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SECTION I. 
Introduction 


A. Load Balance in Cloud Computing 


Cloud computing is an Internet-based resource utility [1], [2]. Its central concept is “everything can 
be a service”. In cloud computing, computing hardware and software resources are capsulized as 
web-services that can be accessed through the Internet. Three application types of cloud computing 
models are defined [3]: infrastructure as a service (IaaS), platform as a service (PaaS), and software 
as a service (SaaS). Among them, virtualization is a representative IaaS application, which provides 
computing infrastructure resources, such as computing power, data storage, networking, all in the 
form of web services [4], [5]. IaaS providers purchase and maintain physical computing and 
storage hardware and provide web services to users. With the virtualization technology, the users 
request IaaS providers for computation or storage resources as they own a “virtual machine”(VM) 
without purchasing and maintaining physical hardware. The users can utilize VMs for deploying 
system/application software with a considerably lower cost of hardware procurement and 
possession. 


An IaaS provider operates a server farm consisting of some computing and storage hardware, 
wherein some hosting servers (referring to as VMHs) provide virtualization services. A VMH may 
run one or many VMs, depending on the capacity of the VMH. If VMHs are not properly managed in 
a server farm, some VMHs may be busy running many VMs but some VMHs almost idle with few 
VMs. Managing loads of VMHs by adjusting the consumption of resources held by VMs for better 
cost-performance efficiency while guaranteeing service level agreements (SLA) to the customers is 


PDF 


an essential issue in IaaS [6]- [9]. 
Help 

Migration of VMs among VMHs is one of the methods for load balance of VMHs, which is to move 

the workload of VMs from one overloaded VMH to other VMHs, trying to make the workload of all 

VMHs evenly. Three phases of work are involved in this process: detection, decision, and action. 

The detection phase is to detect if an imbalance occurs in a server farm. The decision phase is to 

decide if the migration of VMs is needed and select the VMs for migration and VMHs for accepting 
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these VMs. The action phase is to suspend the VMs selected, migrate them from one VMH to others, 


O 
Bana restart them after the migration. During the migration, only the Ms QARtABSA are 


De¥tispended and restarted, and other VMs in the VMHs are stili running. Thus, the workload of 
POWMHs is not static; migration of VMs may cause the workload of the target VMHs to much worse. 


Therefore, an effective load balancing mechanism is essential yet difficult for the management of 
VMHs. 


Load-balance is a typical but essential research topic in parallel or distributed systems, and many 
machine learning-based methods have been proposed, such as [10]— [12]. The following discusses 
some recent methods for load balancing in cloud computing. Alonso-Calvo et al. presented in [13] 
an application-level load balancing system for applications running on VMs. Doong et al. presented 
in [14] a multi-kernel support vector regression model for modeling VMs performance. Hu et al. 
developed a scheduling strategy for load balancing of VM resources using GA that refers to 
historical data and the current state of the system [15]. A capacity allocation algorithm is presented 
in [16] that coordinates multiple distributed resource controllers executing in geographically 
distributed cloud sites. Pang et al. presented in [17] a hybrid method that employs the estimation of 
distribution algorithm to estimate possible solutions of VM loads and then uses GA to adjust these 
solutions. HEELS [18] is a heuristic task deployment approach based on clustering and Glowworm 
Swarm Optimization and used for long-term load balancing of a cloud framework with edge 
computing. A hybrid metaheuristic is proposed in [19] that hybridizing artificial bee colony and ant 
colony optimization for load balancing of VMs in the cloud. 


These methods consider the load balance problem as job-assignment optimization and mainly focus 
on developing optimization algorithms for fast convergence. However, they assume the load of 
VM/VMH is static and do not consider the cost of migration and the load of VMHs after balancing, 
resulting in limited effectiveness in real environments. 

B. Motivation 


Based on the above discussions, an effective and efficient load balancing mechanism must consider 
the following. A load balancing mechanism works in a dynamic environment consisting of many 
running VMs and VMHs. It needs to monitor, detect, and decide how to release the imbalance 
situation of VMHs. An effective load balancing mechanism can take actions that release an 
overloaded VMH and do not result in overloads on another. Also, efficiency and low overhead are 
required for a balancing mechanism. Simple balancing methods like fixed or heuristic-based rules 
work efficiently but not effectively since they may derive overloading of other VMHs. Advanced 
balancing methods like statistics-based can work effectively; however, they need to calculate many 
sophisticated operational parameters and require considerable computation power to decide 
migration targets. 


Because there are diverse IaaS implementations, some assumptions are adopted in this study. 


1) AVM operates on only one VMH at a time, and multiple VMHs share a data storage. The 
amount of resource consumption of a VM cannot exceed the host VMH can offer. 


2) There are many types of load in computer systems, such as CPU/GPU computing load, 
memory/disk storage, network transmission. This study focuses on the CPU load of VMHs, 
which is mainly discussed in the literature. 


3) A centralized load balancing mechanism is employed for monitoring and managing all VMHs. 
The mechanism detects the number, locations, and all VMH workload parameters periodically. 


This study was motivated by the authors’ daily experiences in managing VMHs. Two genetic-based 
methods are developed and integrated for load balance. First, performance models of virtual 
machines (VMs) are extracted from their creating parameters and corresponding performance 
measured in a cloud computing environment. The gene expression programming (GEP) is applied 
for generating symbolic regression models that describe the performance of VMs and are used for 
predicting loads of VMHs after load-balance. Secondly, with the VMH loads estimated by GEP, the 
genetic algorithm considers the current and the future loads of VMHs and decides an optimal 
combination of VM-VMH assignments for migrating VMs and performing load-balance. Our 
proposed methods work effectively and efficiently for balancing VMH workloads in a dynamically 
working VMH environment. The performance of the proposed method is evaluated in a real cloud- 
computing environment. The experimental results show that our method outperforms previous 
methods. 


As stated in the literature, load balance is a critical issue for managing VMH servers in cloud 
computing. Using GEP, the idea of load prediction of VM for optimizing VM-VMH assignment helps 
improve the stability of the load balance mechanism. The white-boxed GEP expression of VM load 


https://ieeexplore.ieee.org/abstract/document/9374470 


TI 


PDF 


Help 


Migration-Based Load Balance of Virtual Machine Servers in Cloud Computing by Load Prediction Using Genetic-Based Metho... 


3/22 


09/06/24, 23:29 Migration-Based Load Balance of Virtual Machine Servers in Cloud Computing by Load Prediction Using Genetic-Based Metho... 


„behaviors can be easily integrated with the balance mechanism. GA provides an intuitive and fast 

method of deciding VM-VMH assignments for load balance and flexibyy RQCAESarious 
ev iective functions for different management purposes. The genetic-based methods, GA and GEP, Q 
PO&an be easily implemented by the cloud administrator. The performance and usability of the 


proposed methods are evaluated and proved to be valid in a real cloud environment. 


TI 


SECTION II. 
Backgrounds 


A. Virtualization 


The virtualization technology is to establish a software layer called a hypervisor on a physical 
hardware platform. A hypervisor is an operating system that protects the resources of a VMH and 
manages the requests and responses from VMs. Depending on the application environments, type- 
1 hypervisors directly running on VMH’s hardware and type-2 running on the VMH’s OS are 
defined. For more details about hypervisors, please refer to [20]. A VM is an abstract machine 
defined by virtual processors (vCPUs), virtual memory (vRAM), and virtual hard drives (vHDs) 
provided by a VMH. A user program running on a VM cannot directly access the physical 
hardware. Instead, all resources are wrapped by the VMH’s hypervisor. Generally, multiple VMs 
are executed on and managed by a VMH. Fig. 1 presents a collection of VMHs (referring to as a 
VMH server farm), where each VMH serves for one or many VMs. Commonly used hypervisors 
include VMware vSphere [21], KVM(kernel-based virtual machine) [22], and Microsoft Hyper-V 


[23]. 


FIGURE 1. - VMs, hypervisor, and a server farm with $m$ VMHs. 


FIGURE 1. 
VMs, hypervisor, and a server farm with m VMHs. 


Conceptually, a VM is composed of a “configuration file” describing the specifications of vCPU, 
vRAM, and vHD and a “disk image file” retaining the user data (vHD). A volatile “memory page” is 
generated in the VMH’s main memory when the VM is running. When a VM is created, the above 
files are created on the VMH’s storage. Therefore, deleting, copying, or moving a VM means 
deleting, copying, or moving these files. One of the advantages of virtualization is the easy migration 
of VMs, which means that one VM can move from one VMH to another VMH. With migration, VM 
can execute on different VMH platforms. During the migration, if the VM is in execution, the VM 
needs to be suspended first. In addition to moving the VM files (configuration and disk image), the 
memory pages associated with the VM must also be moved from the current VMH’s main memory 
to the target VMH. After all the movements are completed, the VM can be activated again. 


During migration, a VM may be suspended for seconds, even dozens of minutes, depending on the 


PDF 


vRAM pages over the network. The larger the vRAM, the more considerable the amount of data to Help 


sizes and storage methods of VHD and vRAM. Additionally, migrating a VM incurs the transfer of 


be transferred, and the greater the system’s burden. The so-called “live migration” [24] is a fast re- 
configuration of VHD and vRAM for migration with a shorter suspending time with which the VM 
users may not even notice the suspension in the migration process. Live-migration is commonly 
implemented with shared storage. VM migration can be easily and quickly done by re-assigning the 
ownerships and disk mapping files instead of physically transferring vVHDs (huge sizes). Fig. 2 
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illustrates the scenario of live migration with shared storage between two VMHs. Suppose the 
administrator selects VM4 to migrate. The disk mapping files associated eantents.. reconfigured 
DoY the storage, and the memory pages are transferred from VMH1 to VMH2. With live migration, 


Q 
PPfoad balance can be more efficient and reduce the impact on working VMs [25], [26]. 
= TT 
#.FIGURE 2. - Live migration with shared storage. 
FIGURE 2. 
Live migration with shared storage. 
gos orig srr race  ama eae 
The load of a VM is dominated by factors like CPU usage, memory size, network speed, etc. Assume 
that v is a VM, and Ly (v) represents its load. The Ly (v) can be described as a function, as shown 
in Eq. (1). 
Ly(v) = f(ur,u2,---, un), (1) 
View Source 
where u1, U2,..., Ug are the aforementioned k factors that dominate Ly . A VMH may serve many 
VMs. The workload of a VMH is the combination of the load from all its VMs and its operating 
system. Assume h is a VMH with n VMs (v;, V2,..., Un ) running on h . The load Ly (h) of h can 
be described as follows. 
n 
Luy(h) = Lo(h) + >) Ly (vi), (2) 
i=1 
View Source 
where, Lo(h) is the system load of h and Ly(v;), 1 < i < n, is the load of the VM v; . Usually, 
each VM’s load is protected by SLA and should be above a reasonable level to maintain good 
service quality. Also, by the quota contract, VM’s load is limited. Eq. (3) describes the upper and 
lower bounds of Ly (v;) . 
l, (vi) < Ly(v;) < i (vi). (3) 
View Source 
Usually, l+ (v;i) and łĮ* (v;) are constants set by the administrator of VMHs according to the SLA and 
contract requirements. If Ly(v;) > l*(v;) , the v; ’s load is too high, and it may not perform the 
user’s tasks smoothly. If Ly (vi) < L (v;) , the load is low, causing a waste of resources of the 
VMH. 
PDF 
Suppose there are p VMHs, hy, . . ., hp , operating in a server farm. The overall load of each VMH Help 


must also be within a reasonable range and can be described as Eq. (4). 


L,(hj) < Lu(hj) < L*(hj), (4) 
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, View Source — 
B := Contents 
Down! 
pprwhere 1 < j < p . Also, L.(h;) and L*(h;) are control parameters set by the administrator. If Q 
Ly(h;) > L*(h;) , the VMH is overloaded, and the VMs running on h; may not obtain sufficient 
resources, degrading the performance and even service quality. If Ly (hj) < L.(hj) , the T 


computing resources are idle and wasted. 


With the above equations, to keep the load of a set of VMHs balanced is to keep all Ly (-) s evenly, 


i.e., 


Liy(h1) = Ly(h2) i] ... © Ly(h,), (5) 
View Source 


and to preserve the consistency of Eq. (4) for all VMHs. If Eq. (5) is corrupted due to the overload 
of some VMHs, some VMs associated with these VMHs may need to be turned off or migrated to 
other VMHs for load-balancing. The load of all VMHs is expected to be balanced after the 
migration of VMs. Suppose h, is a VMH about to become overloaded and h; is a VMH in safe load, 
i.e., Ly(hs) > L*(hs) and Ly(ht) < L* (he) . If migration of q VMs, v1,..., vq , from hs to hy is 
performed, the load of h, and h; changes as follows. 


(hs) =Li(hs) — S Lv (wi); (6) 


Lig (hy) =La (hi) + >) Ly (vs) (7) 


i=1 
View Source 


Therefore, load balancing by the migration of VMs can be considered as a combinatorial problem, 
wherein the best set of VMs are selected from VMHs and migrated to proper hosts and Eq. (4)- 
Eq. (5) are always observed either before or after the VM migration. The following issues are 
considered in this study. 


e The load of VMHs is expected to remain balanced after the migration of VMs. It is necessary 
to predict the load of VMHs to prevent the target hosts from becoming overloaded after the 
migration. One method to obtain the descriptive expression of Ly(v;) in Eq. (2) is to collect a 
sufficient amount of data and find a statistics or regression model accordingly. There are two 
types of learning methods, black-box and white-box [27]. The main difference between them 
is the explainability and readability of the regression model obtained. An explainable VM load 
model can be integrated and adjusted more easily with the management system from the 
system administration perspective. 


e Choosing the best set of VMs and target hosts for migration is a job-assignment optimization 
problem, usually a time-consuming combinatorial explosive problem. An efficient and effective 
selection mechanism is needed for load balance and painless migration. 


SECTION III. 
Genetic Methods 


There are two goals in this study. The first one is to describe the load of VMs as a white-box model. 
The second one is to decide on the best VM-VMH assignment for migration. This research adopts 
two genetic methods, genetic algorithm and genetic expression programming, for achieving these 


goals. 
A. Genetic Algorithm PDF 
The genetic algorithm (GA) [28] is a stochastic search optimization method that mimics natural Help 


genetic systems’ mechanics. With GA, solutions to domain problems are encoded as chromosomes 
that are iteratively processed by genetic operators, reproduction, mutation, crossover, etc. The ones 
with better fitness are retained. GA is well known for its efficient exploitation of better solutions via 
search in the vicinity of known ones. GA has many advantages over traditional search methods: (1) 
it provides a simple and direct representation of solutions; (2) it searches solutions with a 
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„population of potential ones; and (3) it works with probabilistic transition rules instead of 
deterministic ones. GA has been successfully applied to many combi RPA tion problems 
Deytid is widely used in many applications. Genetic programming (GP) [29], is a genetic method for 
PO&volving programs or mathematic formulas. GP uses a tree structure to represent a program or 
formula, as shown in Fig. 3. The genetic operators in GP are similar to those in GA (mutation, 
crossover, and reproduction); however, they are applied as node-renaming, pruning, and 


recombination of subtrees. Because tree structures are complicated and computationally inefficient 


Q 


TI 


for implementation, some improvements are proposed, such as the following. 


gods 


(a) (3x + 8) x (\/y/2) (b)if (a > 3) A(b < 4) then y=5 


FIGURE 3. 
GP trees. 


GA is a type of metaheuristic algorithms that applies specific search schemas for solving complex 
optimization problems. Each metaheuristic algorithm employs a unique representation for problem 
description and a specialized mechanism for exploring the search space and is useful for particular 
applications. GA is adopted in this study due to the following reasons. 


e GA provides a direct and intuitive representation for describing the VM-VMH assignment and 
is easy to implement. 


e Many metaheuristic algorithms are working on an established objective function (fitness 


function), which may be changed for various load balance considerations (see Section VI). 


e For various optimization subjects, only the objective function’s calculation needs to be changed 
without massively changing the gene expression and the whole balance mechanism. 


More comparisons on the difference and usage of metaheuristic methods are available in [30], [31]. 
B. Genetic Expression Programming 


Genetic expression programming (GEP) [32] is a variation of GP. Given a set of terminals (input 
variables or constants) and operators (functions), GEP describes programs or formulas as binary 
trees represented in linear gene expressions. The gene expression is the level-order traversal of a 
binary tree and is composed of head and tail. Symbols in the head can represent functions or 
terminals; the ones in the tail can only be terminals. The length of a gene expression is h + t, 
where A is the length of head given by the users and t = h x (n — 1) + 1 the length of tail. The 
parameter n is the number of most arguments of a function, also given by the users. Referring to 
Fig. 3(a), symbols z, y and integers are terminals and +, —, x , +, ae operators. A GEP tree 
with h = 7 and n = 2 is presented in Fig. 4, wherein the gene expression of length 15 (t = 8 ) 
expresses (3x + 8) x (,/y/2) . Notice that the GEP expression is determined by level-order 
traversal of its binary tree. Some terminals in the tail part may be useless. They are used as 
operands only if their parent nodes are operators. The genetic operators in GEP are reproduction, 
mutation, transposition, and recombination, applying on the gene expression. Reproduction and 
mutation are the same as those in GA. Transposition is to prune a gene segment and insert it to a 
position randomly selected. Recombination is similar to crossover in GA by breaking and 
recombining two gene expressions. The use of expression trees brings efficiency to GEP because 
genetic operations can be more easily applied in a simple linear structure. The cost for search 
specific gene elements in a genetic operation is reduced from O(n) to O(1) . GEP has superior 
performance in dealing with complex problems than conventional regression methods; for PDF 
example, modeling sensor characteristics [33], diagnosis and prediction of lung cancer [34], fault 
diagnosis of power transformers [35], and intrusion detection of power grid [36]. Refer to [32] Help 
for more details about GEP. 
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A .FIGURE 4. - A GEP example-gene and tree expressions. ; = Contents 
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FIGURE 4. 
A GEP example-gene and tree expressions. 


SECTION IV. 
The Proposed Methods 


A. Symbolic Load Models by GEP 


Symbolic regression is to find a description model from the input-output tuples of a problem that 
can rationally predict the correspondence of a new input. By Eq. (1), suppose the load of a VM v is 
determined by k parameters of resource settings, u1, . . -, Up . Let U = {(uij,.--, Uki, Yi) } and yi 
be the load of v , measured with the settings of u1, . . ., ug at the 7 th instance. Suppose 

f (u1,..-, Uz) is a symbolic regression model describing the relation of y and u1, . . ., ug obtained 
from U . If the difference, usually measured by MSE (mean-square error), between f (uri, . - -, Uki) 


and y; is acceptable, the model can be used for describing and predicting the load of VMs. 


MSE(f,U) = D — f (utis. -a uki) )?, (8) 


View Source 


where n is the size of U . When using GEP, terminals are numeric constants and resource 
parameters, U1, ..., Ug , and the operators are mathematic calculations, as listed in Table 1. For 
obtaining a fast VM migration decision, the operators performing complicated calculations (and 


consequently require longer computational times) are assigned lower evolutionary priority. 


TABLE 1 Operators Used in GEP 


Table 1- Operators Used in GEP 


PDF 


Help 
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„B. VM Assignment by GA 


‘= Contents 
ponce there may be many VMs and VMHs working in the cloud, deciding an optimal VM-VMH 


pp@ssignment is combinatorially explosive and time-consuming. Below we show how GA is applied for Q 
fast and reliable VM-VMH assignments. 


TI 


1) Chromosome Encoding 


Suppose that there are q VMs, vo, .. ., Vq—1 , served by p VMHs, ho, . . ., hp—1 , in a VMH server 
farm. A chromosome consisting of q genes is defined as follows. 


[orials (9) 


View Source 


where ¢ , 0 < i < q — 1 , denotes that the 7 th VM is served by the VMH hi; , 
Si € {ho, h2,- - ., hp—1} - A chromosome of the form in Eq. (9) represents a VM-VMH assignment. 
For example, Fig. 5 presents the assignment of 12 VMs to 3 VMHs. 


#-FIGURE 5. - Chromosome encoding for a VM-VMH assignment. 


FIGURE 5. 
Chromosome encoding for a VM-VMH assignment. 


2) Load Estimations 


For the load balancing of VMHs by VM migration, a good configuration should correct the 
currently imbalanced situation. It will not result in a new imbalance after the migration of VMs. 
Suppose the chromosome in Eq. (9) is considered. Let 9 be the VM-VMH assignment before 
migration and € a new assignment generated by GA. Let €(v;) = h; denote the VMH assigned to v; 


in € and é-1(h;) = {v,|€(v;) = hi} the set of VMs assigned to h; in €. By Eq. (2), the load of h; is 
estimated as 


Ly(hi) = Lo(hi) + X` Ly (vy); (10) 
j=l 
View Source 


where vj € €-1(h;) and Ly(v;) = f (u1, . . ., up) is the load predicted by the regression model 
presented in Eq. (8). 


3) Migration Cost 


The cost of migration is also a critical issue that should be considered. Migration is costly, i.e., the 
time/resources consumption for retaining and moving VM data and the profit loss due to service PDF 
suspension. Therefore, for a new VM-VMH assignment, the fewer moving VMs, the better. Let 6 be 


the cost of moving a VM, the migration cost a(£) of a chromosome £ is to examine €(v;) , Help 
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9 <i<q—1, against £o ,ie., a 
‘= Contents 


va a(€) =1- - V yl), a1) 
i=0 
N 0, ¿(vi) = &9(vi), 
os) ie Elvi) A 0(vi), (12) 
View Source 


where 0 < 0 < 1 indicates the impact of migration of a VM and is defined by the system 
administrator. In a configuration with o migrating VMs, the migration cost is 0; otherwise, the 
migration cost is accumulated as the number of migrating VMs increases. Migration of VMs is 
costly, even with shared storage (live-migration). However, migration of VMs is a need for the 
administration of VMH servers from energy-saving or SLA aspects [37]- [39]. A higher 6 setting 
reduces the occurrence of migration and eases the impact or inconvenience resulted from 
migration. Setting a reasonable 6 can refer to the aspects stated in [24], [40]. 

4) Fitness Function 

The fitness of £ is defined by considering the migration cost a(&) and the balance factor (£) . The 
balance factor (£) considers the load difference of two VMHs and computes the balance ratio 


against the maximum workload predicted. 


alé) = n( 5P(P — 1) maxys({Lar(hj)}) (13) 
Svaz Ln (ha) — Lu(ho)| 
View Source 
The overall fitness of € is defined as follows, 
a(é)-B(E), L«(hj) < Lu(hj) < L*(hy), 
fitness(€) = for all j, 
0, otherwise 
(14) 


View Source 


Because the load of VMHs is expected to be “balanced” after migration, the fitness of a 
configuration is bad (0) if any VMH is predicted to be overloaded (> L* ) or underloaded (< Ly, ). 
Otherwise, the migration cost (a(€) ) and the balancing degree (8(€) ) are evaluated. 


C. Load-Balancing Procedure 


Based on the proposed genetic-methods, an intelligent load balancing mechanism is implemented 
with two main procedures. Suppose there are g VMs running on p VMHs, the parameters 
associated with the generation of a GEP-based load model include: historical VM load data or the 
log records periodically collected by VMHs: D , a linear GEP chromosome ( converted from the 
current load model, a set of operators: O , population size: k , best GEP trees to retain: t , MSE 
threshold: e€ by Eq. (8), and head (h ) and the maximum number n of operators in a GEP tree. The 
pseudocode of the GEP-based load model generation is presented in Algorithm 1. The load- 
balance procedure is invoked periodically with the following parameters. The current VM-VMH 
assignment £o = [s0,S1,--+5 Sq-1] in the form of Eq. (9), Load-Balance Status (LB) by Eq. (13), 
Population size: K , Best chromosomes to retain: T , Best fitness threshold: 

B = M x fitness(&) . The pseudocode of our proposed GA-based load balance mechanism is 
presented in Algorithm 2. 


Algorithm 1 The GEP-Based Load Model Generation 
1: function GEP_ Load_ Model_ Generator 


2: Compute tail sizet = h x (n—1)+1; 
3: Evaluate MSE €p of Ço against D ; 


4: while (eo < € ) do 
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5: Randomize k GEP chromosomes ¢; , 1 < i < k, Çi is of length J= t (as in Fig. 4(a)); 
P= S 


onten 


Set the initial population as R = Uk {Gi}; 
Evaluate MSE e(¢;) of ¢; in R by Eq.(8); 
repeat 
Retain top-T elements of R with lower MSE; 
Perform GEP operators on ¢’s; 
Evaluate MSE e(¢;) of ¢; by Eq.(8); 
Select the best element ¢’ as the best VM load model and eg = e(¢') ; 


until (termination conditions: max. iteration or low-enough MSE) 


end while 
return GFP expression of ¢’ as the VM load model; 


end function 


Algorithm 2 The GA-Based Load Balance Procedure 


1: 


10: 


11: 


12: 


13: 


14: 


15: 


16: 


Notice that the current VM load model Ço and the current VM-VMH configuration 9 are also 
include the initial population of GEP_Load_Model_Generator and GA_Balance_Procedure, 
respectively. Such settings make sure that solutions better than the current one can be derived. 


procedure GA_Balance_Procedure 


Evaluate load of each VMH, Ly(h;),0<i<p—1; 
By LB, evaluate load balance status al all VMHs; 


while (LB < o lasts for 30 sec.) do 
Randomize K chromosomes, €; ,1 < i < K , all in the form of Eq. (9); 
Set the initial population as S = Us {é} ; 
Evaluate the fitness of all é; in S by Eq.(14); 
repeat (on S ) 
Retain top-T’ elements with best fitness; 
Perform crossover/mutation on €’s; 
Evaluate the fitness of each €; by Eq.(14); 


until (termination conditions: max. iterations, time limitation, high fitness) 


Select the best chromosome €' as the new VM-VMH assignment if fitness(é') > |B| ; 


Perform migration by €’ ; 


end while 


end procedure 
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„ Initially, the training data for the VM load model can be manufactured, as shown in Section V-B. 
They are collected from VMH’s log and used to update the VM load med Qntents in 
PoS EP Load_Modei_Generator. Therefore, as the system performs, the VM ioad model evoives 
POF-easonably to the real situation. GA_Balance_Procedure performs load balance when an imbalance 
is detected. It searches by GA a new VM-VMH assignment better than the current configuration aT 


and suitably re-balance the system. The two procedures work individually and periodically, as 
depicted in Fig. 6. 


#-FIGURE 6. - Load monitoring and balancing process. 


FIGURE 6. 
Load monitoring and balancing process. 


SECTION V. 
Experiments 


A. Environment and Parameter Settings 


The proposed methods were implemented and verified in a small-scale but real cloud environment, 
Jnet. Jnet is a private intranet, consisting of more than 20 various types of servers accessed by 
about 200 users. The VMHs and the load balance monitor were attached to Jnet. The experimental 
environment consisted of four VMH servers, including three HP ProLiant BL460c blades and one 
HP ProLiant ML350, all running in CentOS. The shared storage server was an HP ProLiant ML110 
running FreeNAS. All machines were connected by 1GB ethernet. KVM is the platform for VMHs. 
The centralized load balancing mechanism was executed and controlled by the VMHo node. The 
architecture and specifications are shown in Fig. 7. Because the resource capacity of each VMH is 
different, the number of VMs that a VMH can support is also different. In the following 
experiments, the VM capacity of VMHo, VMH1, and VMH2 is 48, and the capacity of VMH3 is 12. 
There are in total 48 VMs running in each experiment. 


# FIGURE 7. - Experimental environment settings. 
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FIGURE 7. 
Experimental environment settings. 
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PDFnumber of resources used by VMs. Such parameters are contract-protected and adjustable. In this Qa 
study, two parameters of CPU utilization are discussed. T 
T. 
e CPU utilization: This parameter explicitly limits the CPU consumption of a VM and restricts 
the VM’s performance/load. There are various implementations of such parameters on 
different virtualization platforms. The one used in this study is cpu.cfs_quota_us defined by 
the Linux Cgroups [41], which ranges from 1000 to 100000, representing the CPU utilization 
from 1% to 100%. 


e Usage priority: This parameter defines the priorities of VMs for using a CPU core. Also, the 
parameters cpu.shares defined be the Linux Cgroups is used in this study, which ranges from 1 
to 65535. 


This study defines two parameters for evaluating the performance of the load balancing methods. 
e Load-Balance status (LB). LB is the value of Eq. (13) applied to the current VM-VMH 
configuration, the more balanced load of VMHs, the higher LB. 


e Balancing Efficiency (BE). BE is the number of load balancing iterations performed by the 
monitor to make all VMHs balanced (LB>o) after migration of VMs; the fewer rounds, the 
faster the load balancing efficiency. 


The balancing and monitoring mechanisms were implemented in C++ with libvirt [42] as the 
software interface. Each VMH scans and logs its CPU load by Eq.(2) every 10 seconds. A load 
imbalance situation is detected if LB<o lasts for the past 30 seconds. Load limitations of a VM are 
L* = 25 % and L, = 0 %. The length of a chromosome is the number of VMs currently served in 
the data center. In the following experiments, six load balancing heuristics are compared with our 
proposed one. 


1) LtoS: move the top-loaded VM from over-loaded VMHs to the VMH with the lowest load. 

2) LtoR: move the top-loaded VM from over-loaded VMHs to a random VMH. 

3) StoS: move the lowest loaded VM from over-loaded VMHs to the VMH with the lowest load. 
4) StoR: move the lowest loaded VM from over-loaded VMHs to a random VMH. 

5) RtoS: move a random VM from over-loaded VMHs to the VMH with the lowest load. 


6) RtoR: move a random VM from over-loaded VMHs to a random VMH. 
B. Modeling VM Loads 


The first experiment is to derive load models of VMs. For training GEP, a set of data in the form of 
( cpu.cfs_quota_us, cpu.shares, Ly (-)) was collected. Because there is no historical VM load data, 
we have to generate load data similar to real VMs. A VM v* was designed for collecting load data 
with a load generator. The generator performed several computation-intensive and I/O-intensive 
instructions. The computation-intensive part is to calculate sin((double)rand()/RAND_MAX) and 
the I/O-intensive part is to read data from /dev/zero and write to /dev/null. The program was 
randomly executed and lasted for 10 seconds, the computation or I/O instructions on v* . The 
parameters, cpu.cfs_quota_us and cpu.shares were randomly set for v* ; cpu.cfs_quota_us was set 
10000-100000 and cpu.shares was partitioned into 1024 levels. A total of 1024 combinations of 
parameters were generated, each used to initialize v* . The VM v* executed on a VMH for seven 
times, and the load Ly (v*) was measured. A total of 7168 load data were collected. Then, the 
operators listed in Table 1 and the hyper-parameters listed in Table 2 were used with terminals 
cpu.cfs_quota_us and cpu.shares for symbolic regression of Ly(v*) . 


TABLE 2 GEP Hyper-Parameters 
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To verify the effectiveness of the GEP models, some regression methods [43], polynomial(POL), 
linear(LIN), exponent(EXP), and power(POW) regression, were used to analyze the same VM load 
data and compared it with GEP. Variable portions of the training data were randomly selected for 
training, and the rest portions were used for validation. The same training process was repeated ten 
times. The training results are presented in Fig. 8. 


FIGURE 8. - GEP training results with 10%-90% training data. 


FIGURE 8. 
GEP training results with 10%-90% training data. 


Because the best model was with 90% of training data, the same data set was also applied to other 
regression methods. The load models generated by various regression methods are as follows: 


e POL: —0.166 + 0.0939u; + 0.0001u? — 0.00004u> + 0.00000001u2 


e LIN: —0.3235 + 0.103u; — 0.00002u9 


EXP: exp(—0.4866 + 0.0337u; — 0.00004u2) 


e POW: 0.0394ut 7216, 0-0004 


GEP: In((1/uz) — 0.06u2) 
PDF 
For simplicity, in the above models, cpu.cfs_quota_ us $= u_1$ and cpu.shares $= u_2$. Fig. 9 
and Table 3 present the testing results of all regression models. In terms of MSE and MAE, GEP, Help 
POL, and LIN perform competitively, and EXP is the worst. MAPE indicates that GEP, POL, and 
POW are more stable than others. POL and GEP produce similar error rates; however, POL uses 
more and complicated operators than GEP does. LIN uses the fewest operators but is more 
unreliable in terms of MSE/MAE and unstable in MAPE than GEP. GEP produces a much compact 
formula than others and can be considered competitive. 
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# FIGURE 9. - Statistics on errors of all models. 


FIGURE 9. 
Statistics on errors of all models. 


C. Load Balance Scenarios 


In the following experiments, the VM load model generated by GEP was used for load prediction. 
The load balancing monitor of Fig. 6 was applied to four scenarios. A total of 48 VMs were 
deployed, each of which performed similarly as v* . In the following tests, 40 runs of load balancing 
were observed. VM migration was performed when the load was imbalanced, or the GA produced a 
VM-VMH assignment with LB 1.25 times better than the current configuration. The hyper- 
parameters associated with GA include: max. generation: 1000, population size: 30, 
crossover_rate: 0.7, mutation_rate: 0.4, time limitation: 60 sec. 


1) Scenario A 

In this scenario, an extreme imbalance is simulated. In the beginning, all VMs were gathered 
together on a randomly selected VMH, making the VMH heavily overloaded. Fig. 10 plots the 
change of load of all VMHs in 40 rounds of load-balancing. Table 4 lists the loads of all VMHs, 
where R2R is the average of RtoRi and RtoR2 ( Fig. 10(g)(h)). From Fig. 10, it is observed that in 
the first round, all VMs are concentrated to a VMH, the VMH loads are extremely imbalanced, and 
thus, the LB value is o. However, our proposed method takes only one round to re-balance the loads 
for all VMHs. The workloads of all VMHs approximate evenly after load-balancing, which can be 
observed in Table 4. Other strategies take more iterations for re-balancing the loads. In the cases 
using LtoS, StoS, and RtoS, LtoS migrates the most heavily loaded VMs to the lowest loaded VMH 
and reaches a balanced state in 20 rounds, causing load vibration before back to balanced. Though PDF 
RtoS takes 35 rounds for re-balancing and StoS even not back to balanced, the load of VMHs trend 


to balance. Other strategies even not back to the balanced state. Help 


TABLE 4 Comparisons on Average Load (%) in Scenario-A 
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>FIGURE 10. - Loads and LB trends in Scenario-A. 


FIGURE 10. 
Loads and LB trends in Scenario-A. 


Another extremely imbalance is studied in this experiment, wherein a temporary service suspension 
of a VMH happens so that all its VMs have to be migrated to other VMHs for non-stop services. In 
this scenario, all VMs on VMHi are moved to VMHoO at round o, and VMHi backs to service at 
round 1. The balancing status is observed in Fig. 11. It is found that VMHo is overloaded, and VMH1 
is idle in the beginning. Again, our proposed method takes only one round to re-balance loads of all 


VMHs and keep them almost balanced in the consequence rounds. Some strategies can also re- 
balance the loads, such as LtoS, LtoR, StoS, and even RtoR; however, they take more rounds to re- 
balance and only manage a low-level balanced status. This phenomenon can be observed in Table 5. 


TABLE 5 Comparisons on Average Load (%) in Scenario-B 


Zable 5- Comparisons on Average Load (%) in Scenario-B 
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FIGURE 11. 
Loads and LB trends in Scenario-B. 


3) Scenario C 

This experiment simulates the daily use of VMs-—the turning on/off of VMs at peak/non-peak 
hours, wherein more VMs are online/offline in the peak/non-peak hours. Thus, VMHs are less- 
loaded at non-peak hours and vice versa. The timing of turning on or off a VM is decided by a 
Gaussian distribution model, with 1 = 10 and ø = 5 for turning on VMs and u = 20 ando = 5 
for being online before turning off. That is, a VM starts at around the 10th round and stays online 
for about 20 rounds, simulating the 20th round as the peak hour and the 1st and 40th rounds the 
off-peak hours. The results are presented in Fig. 12. Notice that, due to no fair comparison on two 
VMH load that is stochastic in this scenario, average load of VMHs is not presented. Obviously, the 
VMH load in this scenario is highly dynamic, which challenges the load-balancers. In the 
beginning, all methods do not balance VMH load effectively due to the unpredictable behaviors of 
VMs being online, and all LBs are o. It is found that all methods can balance loads to some extend 
in peak hours. This may be because the load behaviors become stable in the peak hours, with more 
VMs being online so that the load status is more easily to remain after VM migration. The LtoR 
strategy is the first to re-balance the load; however, it only works in the non-peak hours (the 5th 
round) when there are only a few VMs. Some strategies, including LtoR, LtoS, StoR, do not balance 
loads effectively in the peak hours and even failed after the peak hours. Our proposed method 
demonstrates a better performance on load balancing than others. 


FIGURE 12. - Loads and LB trends in Scenario-C. 


FIGURE 12. 
Loads and LB trends in Scenario-C. 
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4) Scenario D Help 


Scenario D is an extended version of Scenario C, wherein the dynamics of VMHs is considered. In 
this experiment, the system load of a VMH, i.e., Lo(-) , bursts intentionally, and the VMH is 
undoubtedly overloaded. Thus, the VMs served by this VMH have to migrate for load-balancing. 
Additionally, VMs behave like those in Scenario C; they turn on and off in peak/non-peak times. 
Two VMHs were randomly selected as the load-burst hosts at the 7th and 27th rounds, 
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„ Tespectively. Their load bursts continue for seven rounds and back tq service at the 14th and 34th 
rounds, respectively. The balancing results are presented in Fig. 13..Oer FERkePtiethoa is 
De*Ssnsitive to the imbalance in this highly dynamic environment and can re-baiance the VMH loads Q 
POFefficiently. Static heuristic rules, such as LtoS and StoS, are also sensitive and able to re-balance; 
they do not work well in all cases. Random heuristic rules may be workable; they are not stable and aT 


have poor performance. 


‘FIGURE 13. - Loads and LB trends in Scenario-D. 


FIGURE 13. 
Loads and LB trends in Scenario-D. 


Table 6 lists the averaged LB and BE values of all methods in the four scenarios. A higher LB value 
means a better-balanced performance. A lower BE value presents better balancing efficiency. Our 
proposed method demonstrates effectiveness and efficiency in either stable (e.g., scenarios A and B) 
or dynamic (e.g., C and D) environments in terms of LB and BE. With VM load models obtained 
from GEP, our method predicts the VMH loads after migration and computes by GA the most 
effective VM-VMH assignment for migration. Other strategies perform heuristic VM migration and 
take longer rounds for re-balancing, or even not back to balance. Interestingly, according to LB and 
BE in Table 6, LtoS is the second-best strategy. LtoS is intuitive and straightforward to implement, 
i.e., to move the worst VM to a VMH with the lowest load; however, it takes a longer time to re- 
balance VMH loads. Random methods, such as LtoR, StoR, RtoS, and RtoR, are effective sometimes 
but are not recommended due to their stochastic characteristics and unstable performance. Our 
proposed method performs effective and efficient load balancing in all tests. It is competitive and 
more promising than others. 


TABLE 6 Comparisons on LB and BE of all scenarios. 


Table 6- Comparisons on LB and BE of all scenarios. 
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GEP_Load_Model_Generator updates the VM load model every 120 seconds periodically. The 
monitor GA_Balance_Procedure is activated every 250 seconds and calculates the LB value of all 
VMHs. Notice that the frequency of executing the two procedures should not be too high. Higher 
updating and balancing frequency may keep the VM load model and the system as real and balance 
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„as possible. However, the operations of performance measurement and migration are costly: the 
BSsteal of CPU cycles degrades the overall performance and QoS, wheres fe Rabe Atsration may 
De¥aiuse unacceptable service suspension. Hence, the period of load monitoring is 250 seconds and the Q 
POF eriod of updating the VM load model is 120 seconds. 
Moreover, as in other machine learning algorithms, setting hyper-parameters associated with GA i 
and GEP is subtle and complex. The size of population, probabilities of genetic operators, and the 
termination condition are all problem-dependent. In general, higher probabilities of invoking 
genetic operators usually drive GA or GEP early mature but easily trapped in local optimal; lower 
probabilities take much time to converge. A large population consumes considerable memory space 
and computing time but may earn a higher chance of evolving a better solution. Due to the costs of 
performance probing and migration, deriving a solution (VM load model or VM-VHM assignment) 
should be timely. Such a solution may not be the best one but has to be acceptable and better than 
the current one. For example, a new VM-VMH assignment has to be 1.25 times better than the 
current one to be acceptable. Other hyper-parameters are set by observing the behaviors of VMs and 
VMHs in JNet. Several combinations of parameters are attempted. For the length of this paper, 
some meaningful parameter sets are presented in the experiments. For issues of setting hyper- 
parameters associated with genetic methods, the readers can refer to [44], [45]. 


Regarding the efficiency of the balance mechanism, the main concern is how fast an effective VM- 
VMH assignment is suggested. The complexity of each strategy depends on how it is implemented. 
In this study, all strategies are implemented with standard C/C++ libraries. Searching for a 
min/max item can be done in O(n) and generating a random number can be done in O(1) , where 
n is the number of VMs. However, running a GA-based method is stochastic. Its complexity 
depends on the representation of chromosomes, the implementation of the genetic operators, and 
the fitness function. Some studies presented theoretic and experimental analysis on the complexity 
of GA [46], [47]. The complexity of our method is more likely O (gpn) for the stochastic process 
and O(n?) for fitness evaluation, where g is the number of generations and p is the population 
size. In our experiment, g is 1000 and p is 30. The GA iteration is usually terminated by a low LB 
value and is less than g . When implementing our method in a large-scale cloud environment, the 
fitness evaluation could be the computational bottleneck, which can be implemented with a 
sophisticated data structure to reduce the computation cost. 


SECTION VI. 
Conclusion 


This paper presents a load balancing mechanism based on evolutionary computing. Loads of VMs 
and the associated resource parameters are measured and used for constructing symbolic regression 
models of VM using GEP. An optimal combination of VM-VMH assignment is decided by GA, which 
predicts VMH loads by GEP models and suggests the VMs be migrated for load-balancing. 
Experiments are conducted in a small-scale but real cloud environment. The proposed method 
demonstrates its effectiveness and efficiency on load balancing and is competitive and promising. 
Although the proposed method performs well for load-balance, it can be improved in some aspects 
and discussed. 


Currently, constant lengths of the head and tail of a GEP chromosome may limit the searchability of 
GEP in the solution space. The variable size of the head that is determined by the training data may 
produce better regression models. The initial VM load model is built from simulated data. However, 
according to the architecture depicted in Fig. 6, VM load data are collected continuously; the VM 
load model can also be updated accordingly. GEP and GA are the genetic-based methods employed 
in this study. It is possible to use other methods to generate regression models and decide the 
optimal combination of VMs for migration, provided that white-box regression models and fast 
decision-making can be derived. 


Due to the hardware resource limitations, the test environment only consists of four VMHs with a 

centralized load-balancer installed on one VMH. However, the usability of our proposed method is 

verified. The proposed method can be applied to a distributed, large-scale management architecture 

easily. At present, the CPU load of VMs and VMHs is considered. Other types of load, like PDF 
networking bandwidth, memory data transfer rate, or GPU loads, may also be worth studying. Help 
However, these types of loads are stochastic, and their measurements take considerable overhead. 


They are not included in this paper but may be processed similarly by our proposed method. 


This study focuses on the load balancing of VMHs that demand VHSs to work evenly. There may be 
other VMH management strategies, e.g., power saving or utilization maximization, that can be done 
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„by load consolidation, i.e., to aggregate VMs to some high-performance or energy-efficient VMHs so 
Binat some VMHs can be turned off [48]- [50]. These can be easily dore uG POSEN posed 
De¥ethods by re-defining the fitness function of GA. The current fitness function and balance factor 


Q 
POfire based on the ratio of the maximum load to the load differences among VMHs, which is simple 
but fast. Comprehensive statistics on the VMH loads may produce a sophisticated fitness function T 
T. 
and LB formula [51]. Considering all these factors for load balancing is a multi-objective 
optimization problem, which requires sophisticated methods for maintaining computation 
efficiency. However, measurements on the detailed resource consumption of VMs may take much 
more computational resources and thus degrade the performance of VMHs and the load-balancer. 
The trade-offs between efficiency and measurement need advanced studies. Instead of GA, other 
metaheuristic algorithms may also decide VM-VMH assignment efficiently. GA is competitive than 
others in the practical administration of cloud computing servers because it can easily and flexibly 
adjust the optimization objectives and maintain a clear and intuitive representation for VM-VMH 
assignment. Studies on the performance and applicability of various metaheuristic algorithms for 
different balance purposes are worthy. These will be part of our future work 
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